Electronic cameras, equipped with only one matrix image sensor (e.g., a CCD image sensor) for taking images and which nevertheless produce a color image with a single exposure, exist both in the form of video cameras (moving picture) and electronic digital cameras (still image). A mosaic filter (CFA=color filter array) allows such cameras to differentiate between colors. Such a filter is placed directly on the surface of the CCD image sensor while each cell (pixel) of the CCD image sensor is covered by a filter element (pixel filter) having suitably chosen spectral transparency.
The document "The Seybold Report on Publishing Systems"(12 Dec. 1994) discloses a digital camera which can take images with a single exposure of a CCD image sensor. The individual cells of the CCD image sensor are covered by four different filter elements and the filter elements are placed directly on the individual cells of the CCD image sensor. The four filter elements employed for taking an image are red, green, blue and blue-green. The filter elements are arranged on the CCD image sensor in a mathematical pattern. In order to determine the color of each individual pixel of the CCD image sensor, the color values are first determined by an algorithm over an area of 64 pixels and afterwards each pixel is compared with its nearest neighbor. If this is not sufficient to determine the color, then the comparison with the nearest neighbors is extended to a larger area. In transition regions from one color to another, up to one thousand calculations are required for one pixel. Since the data are not available in standard RGB format, a conversion has to be performed which takes a high performance computer approximately 7 minutes per image to complete.
Video camera also exist which have combinations of additive and subtractive filter sets, such as, for example, green, cyan, magenta, yellow or green, white, cyan, yellow (see EP-A-0 570 204). Such combinations are intended to better adjust the spatial scanning frequencies for luminance and chrominance signals to the bandwidths of the video system and hence, a reduction of the color Moire effects. However, such combinations do not achieve, nor are they aimed at improving color fidelity or improving the spectral selectivity of the image. In the video systems discussed in EP-A-0 570 204 the signal processing is performed in "real time" without intermediate storage of images. Thus, the technical set-up is completely different than in the case of digital still image cameras as referred to in the process according to the present invention.
Most electronic color cameras have mosaic filters with three different colors. In almost all electronic cameras these different colors are arranged in rows and columns in a periodically repeating pattern. The pattern created by repetition of a (2,2) matrix with identical diagonal elements is known as a Bayer pattern and is widely used as a mosaic color filter. Highly developed processes for color interpolation in mosaic filters with RGB filters in the Bayer pattern (U.S. Pat. No. 5,373,322 and U.S. Pat. No. 5,382,976) exploit the peculiarities of this pattern in a decisive way, that is, they use the dominance of green and the low incidence of fuzziness in the case of red and blue. In addition, the processes disclosed in U.S. Pat. No. 5,373,322 and U.S. Pat. No. 5,382,976 have been developed in such a way that regardless of the contents of the image, the process is able to decide whether more use should be made of the row-wise or of the column-wise correlations of the image data. This produces texture-like artifacts in those image areas which have no details, but which are noisy.
An electronic camera has to be able to produce image data sets in a conventional format, i.e., TIFF or TARGA. This means that a value has to be provided for each pixel and for each color belonging to the data set. Thus, for each pixel an R-value, a G-value, and a B-value have to be available in the RGB data set. It is evident that the raw data of the camera do not satisfy this requirement: they contain only one color value for each pixel, that is for a color which changes from pixel to pixel. Hence, the camera must have a means by which it can use the raw data to calculate these missing color interpolation of mosaic filters or, in short, a process for color interpolation.
The intention is to arrive at values which would be obtained when taking images without using a mosaic filter at arbitrarily short time intervals with the different filter elements nF and using a filter wheel which contain the nF filter colors as filters covering the overall surface area.
The quality of such a process is a complicated matter. As the lack of object information is at best replaceable by plausible, but never by completely certain assumptions, it is always possible to construct instances of images whose quality would seemingly indicate that the process is a failure. For example, it is easy to see that for each set of raw data there is a distribution of white light which would generate these raw data. On the other hand, there are often plausible reasons why such phenomena need not be taken into consideration. The most noticeable artifacts ("aliasing") occur in the case of images in which the image signal differs greatly from pixel to pixel. In most electronic cameras with relatively few pixels, a "blur filter" ensures that no such finely structured light distributions can occur in the area of the sensor.
Distinctions are made between the following, partly overlapping quality complexes:
1. Faithfulness to detail: the ability to reproduce fine structures which extend only over a few pixels, the reproduction being made without greatly changing the form. When comparing two different systems, improved faithfulness to detail is evidenced by improved legibility of the text, the elements of which only cover a few pixels. PA1 2. Artifacts: these include all image structures of which, without possessing detailed knowledge of the camera, it can be said that they are very probably not the same as the original. Examples are: PA1 3. Color fidelity: considered here for monochromatic areas which extend over many pixels and in all directions. PA1 storing values of each pixel of the CCD image sensor in a dedicated level of an image memory, said values being measured by the CCD image sensor and being transformed by a following analog/digital converter into digital values; PA1 deriving an "unfocussed image" so that for each color f and for each pixel p, the neighboring pixels of color f are determined, their values read from Bo and combined to yield weighted average values; PA1 storing the values for all the different filter elements on the CCD image sensor in dedicated image memory levels; PA1 calculating the final color values by means of additive image sharpening and PA1 storing the final color values in the dedicated image memory levels.
a) color edge artifacts: strongly colored striped pattern in the transition area between evenly, but differently colored areas. PA2 b) color Moire: semi-even, usually striped color variations over periodically structured areas. PA2 c) flat-field artifacts: textures, defects and jumps in intensity along the lines of such image areas which apparently lie outside the focal area of the optical image and which, thus, do not show any sharp differences in intensity.